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ABSTRACT 

This paper considers applications of decision theory 
to the problem of instructional decision-making in computer-based 
adaptive instructional systems, using the Minnesota Adaptive 
Instructional System (MAIS) as an example. The first section 
indicates how the problem of selecting the appropriate amount of 
instruction in MAIS can be situated within the general framework of 
empirical Bayesian decision theory. The linear loss model and the 
classical test model are discussed in this context. The second 
section describes six characteristics essential in effective 
computerized adaptive instruccional systems: (1) initial diagnosis 
and prescription; (2) sequential character of the instructional 
decision-making process; (3) appropriate amount of instruction for 
each student; (4) sequence of instruction; (5) instructional time 
control; and (6) advisement of learning need. It is shown that all 
"DUt the sequence of instruction could be improved in MAIS with the 
extensions proposed. Several new lines of research arising from the 
application of psychometric theory to the decision component in MAIS 
are reviewed. Thirty-six references and a list of 28 recent research 
reports of the University of Twente (The Netherlands) Division of 
Educational Measurement and Data Analysis are included. (MES) 
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Abstract 



The purpose of this paper is to consider tome applications o£ 
decision theory to the problem o£ instructional decision 
making in computer-based adaptive instructional systems. In 
particular, the Minnesota Adaptive Instructional System 
(MAIS) will be dx^scussed. It will first be indicated how the 
problem of selecting tho appropriate amount of instruction in 
MAIS can be situated within the general framework of 
(empirical) Bayesian decision theory. Subsequently, it will 
be shown how five out of six characteristics identified as 
essential in an effective computerized adaptive instructional 
system can be improved by extending MAIS in two directions 
(viz. adopting a linear loss model and the classical test 
model) . 
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Introduction 

Computer-based adaptive instructional systems were been 
studied recently by Atkinson (1976). Ferguson (1970). 
Hambleton (1974) . Hansen. Ross and Rakow (1977) . Holland 
(1977). Park (1982) and Tenny5?on and Breuer (1984). Although 
different authors have defined the term "adaptive 
instruction" in a different way. most agree that it denotes 
the use of strategies to adapt instructional treatments to 
the changing nature of student abilities and characteristics 
during the learning process (Tennyson & Park. 1984). 

Four empirically based adaptive instructional modes have 
been reviewed by Tennyson and Park (1984). The four models 
are Atkinson's mathematical models. Ross's trajectory model. 
Ferguson's testing and ioranching model, and Tennyson's 
Minnesota Adaptive Instructional System (MAIS). These four 
models vary in degree to which they use six characteristics 
identified as essential in an effective adaptive 
instructional system. The authors conclude that MAIS provides 
for a complete adaptive instructional model, because all six 
defined characteristics of adaptive instruction are 
integrated into this model. 

The purpose of this paper is to generalize and extend 
the application of Bayesian decision theory in MAIS. First, 
it will be indicated how this model can be situated within 
the general framework of (empirical) Bayesian decision 
theory, and what implicit assumptions have to be made in 
doing so. Subsequently, it will be shown how five out of the 
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six defined characteristics of computerized adaptive 
instruction in the MAIS model can be improved by applying 
other results from decision theory to this nodel. A linear 
loss function is proposed to replace the threshold loss 
function assumed in HAIS. Moreover, optimal sequential 
decision rules will be derived using Eelley's regression line 
of classical test theory as the psychometric model instead of 
the binomial model assumed in MAIS. The paper concludes with 
a discussion of some new lines of research arising from the 
application of decision theory to the MAIS model. 



Adapting the Amount of Instruction 

Initial work on MAIS began as &n attempt to design an 
adaptive instructional strategy for concept-learning 
(Tennyson, 1975). According to Merrill and Tennyson (1977), 
concept-learning can be conceived as a two-stage process of 
formation of conceptual knowledge and development of 
procedural knowledge (for a complete review of the theory of 
concept-learning, see Tennyson and Cocciarella, 1986). 

In MAIS, eight basic instructional design variables 
directly related to specific learning processes are 
distinguished. In order to adapt instruction to individual 
learner differences (aptitudes, prior knowledge) and learning 
needs (amount and sequence of instruction), these variables 
are controlled by an intelligent tutor system (Tennyson, 
Christensen & Park, 1984). The authors consider MAIS as an 
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intelligent system , because it exhibits some machine 
intelligence, as demonstrated by its ability to improve 
decision making over the history o£ the system as a function 
of accumulated information about previous students. In the 
literature, successful research projects on MAIS have been 
reported (e.g.. Park & Tennyson, 1980; Tennyson, Tennyson & 
Rothen, 1980). Three out of the eight instructional design 
variables are directly managed by a computer-based decision 
strategy, namely amount of instruction, instructional time 
control, and advisement on learning need. These three 
variables also belong to the six characteristics of 
computerized adaptive instruction. In HAIS, selecting the 
appropriate amount of instruction can be interpreted as 
determining the optimal number of interrogatory examples 
(question form) . 

The derivation of an optimal strategy with respect to 
the number of interrogatory examples in a concept— learning 
le3Son requires an instructional problem be seated in a form 
amenable to decision-theoretic analysis . In the Bayesian view 
of decision making, there are two basic elements to any 
decision problem: a loss function describing the loss l(aj^,ir) 
incurred when action aj^ is taken for the student whose true 
level of functioning is ir (0 ^ ir ^ 1), and a probability 
function or psychometric model, f(x|ir), relating observed 
test scores x to student's true level of functioning. 

These basic elements have been related to decision 
problems in educational testing by many authors, particularly 
in the context of computer-based adaptive instructional 
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systems (e.g., Atkinson, 1976; Swaminathan, Hambleton & 
Algina. 1975; van der Linden, 1981a). As the use of the 
decision component in MAIS refers to sequential mastery 
testing, we shall discuss here only the application of the 
basic elements to this problem. 

It is assumed that , due to measurement and sampling 
errors, the true level of functioning ir is unknown. All that 
is known is the student's observed test score X from a small 

sample of n interrogatory examples (x = 0,1 ,n). 

Furthermore, the following two actions are available to the 
decision-maker: advance a student (a^) to the next concept if 
his/her tesc score X = x exceeds a certain cut-off score c on 
the observed test score scale X, and retain (ag) him/her 
otherwise. Students with test score X below the cut-off score 
c are provided with additional expository examples (statement 
form). A new interrogatory example is then generated. This 
procedure is applied sequentially until either mastery is 
attained or the pool of test items is exhausted. Now, the 
sequential mastery decision problem can be stated as choosing 
a value of c that, given the value of the criterion level ttq. 
is optimal in some sense. The criterion level itq € [0,1] - 
the minimum degree of mastery required - is set in advance by 
the decision-maker. 

Generally speaking. ?v loss function specifies the total 
costs of all possible decision outcomes. These costs concern 
all relevant psychological, social, and economic consequences 
which the decision brings along. An example of economic 
consequences is extra computer time associated with 
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presenting additional instructional materials. In Tennyson's 
approach, the loss fxmction is supposed to ho a threshold 
function. The implicit choice o£ this function implies that 
the "seriousness'* of all possible consequences of the two 
available actions can be summarized by four constants , one 
for each of the four possible decision outcomes (see Table 



For convenience, and without loss of generality (e.g., 
Davis. Hickman & Novick, 1973). it is assumed in Table 1 that 
no losses occur for correct decisions, and. that therefore, 
the losses associated with correct advance and retain 
decisions (l^i and Iqq. respectively) can be set equal to 
zero. 

In the decision component of MAIS, a loss ratio R must 
be specified. R refers to the relative losses associated with 
advancing a learner whose true level of functioning is below 
itq and retaining one whose true level exceeds itq. From Table 
1 it can be seen that the loss ratio R equals Iiq/Iqi for all 
values of ¥. 

Finally, it is assumed that the psychometric model in 
MAIS relating observed test scores X to the true level of 
functioning ir can be represented by the binomial model, 



1). 



Insert Table 1 about here 



ERIC 




Applications o£ Decision Theory 

7 

(1) £(x|Tr) = (J) ir3C(l-Tr)a-3C. 

Within a Bayesian decision-theoretic framework a 
sequential mastery decision problem is solved by minimizing 
the *'Bayes risk** , which is minimal i£ for each value x of Z 
an action with smallest posterior expected loss is chosen. 
The posterior expected loss is the expected loss taken with 
respect to the posterior distribution of ir. 

It can be seen from the loss table that a decision rule 
minimizing posterior expected loss is to advance a student 
whose test score x is such that 

(2) l^QProbCir < Tro|x,n) £ iQiProbdr > irolx.n), 

and to retain him/her otherwise. Since Iqi > 0, this is 
equivalent to advancing a student if 

(3) Probdr ^ irolx.n) ^ 1/(1+R). 

and retaining him/her otherwise. Prob(ir ^ iro|x.n) denotes the 
probability of the student's true level of functioning 
smaller or equal to given a test score x on a test of 
length n. In fact» this probability is given by the 
c\;imulative posterior distribution of ir. In MMS this quantity 
is called the "beta value" or "operating level" (Tennyson. 
Christensen & Park, 1984). 
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It should be noted that, as can be seen from the 
decision rule, the declslon<-aaker need not specify the values 
llQ and Iqx completely. He needs only assess their ratio 
I^q/Iqi. For assessing loss functions, most texts on decision 
theory propose lottery methods (see, for example. Luce & 
l^alffa, 1957, chap. 2; for a recent modification, see Kovlck 
& Llndley, 1979). But In principle any psychological scaling 
method can be used. Although helpful techniques are 
available, this does not me€Ui that, for example* in programs 
of individualized instruction, assessment of losses is always 
a simple matter. In the next section, we shall consider one 
method that works in decision problems with a finite number 
of outcomes such as the sequential mastery decision problem. 

In order to initiate the decision component in MAIS, 
three kinds of parameters must be specif^dd in advance 
(Rotnen & Tennyson, 1984). Beside the parameters itq and R, a 
probability distribution representing the prior knowledge 
about ir must be available. In MAIS, a beta distribution, 
B(a,P), is used as a prior distribution, and a pretest score 
together wil-n information about other students is used to 
specify its parameter values. 

Seats and Lord (1962) have shown that simple moment 
estimators for a and P can be derived that are based on the 
mean, the standard deviation, and the ICR-21 reliability 
coefficient of the test scores from the previous students. 
Let the 0-21 reliability be defined as 



13 



Applications of Decision Theory 

9 



n jijdi-nx) 

(4) r2j = [1 5 ]. 

n-l aah 

where and o^Z denote the mean and the variance of the 
pretest scores, respectively. Then the estimates o and P of 
a and respectively, are given as 

(5) S = (-1 + l/r2i)^x 
P = -a + n/r2i - n- 

It follows that the posterior distribution of ir is 
easily obtained. From an application of Bayes' theorem, the 
posterior distribution will again be a member of the beta 
family (the conjugacy property). In fact, if the prior 
distribution is B(a,P) and the student's test score is x from 
a test of length n, then the posterior distribution is 
B(x-^a,n->x-^P) . The beta distribution has been extensively 
tabulated (e.g., Pearson , 1 930 ) . Normal approximations are 
also available (Johnson & Kotz, 1970, sect. 2.4.6). In 
general, if if has a beta distribution with parameters (a,P; 
where neither a nor p is small (say, not < 10), then this 
distribution can be approximated by a normal distribution 
with mean a/(a+P) and variance ap/((a+P)2(a+p+l) ) . These mean 
and variance are Just those of the beta distribution. 
Tennyson and Christensen (1986) use a non-linear regression 
approach that fits the best polynomial as an approximation of 
the beta distribution. Finally, Vijn and Molenaar (1981) have 
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shown, in the context of robustness regions for dichotomous 
decisions, that, if we put 



tjj - .4a+4p+4n+2)"''. 
expression (3) can be replaced by 
(7) TQ - gN ^ -(tN/1.7)lnR, 

from which the optimal cutting score x can easily be 
evaluated on a computer. 

The MAIS decision procedure for adapting the number of 
interrogatory examples can now be siunmarized as follows: 7.f 
the quantity l/d-fR) exceeds a student's beta value, (s)he is 
passed to the next instructional unit (i.e., next concept) or 
final (sunmative) posttest. However, if his/her beta value is 
below this quantity, his/her posterior distribution is used 
as a prior distribution in a next cycle. A new interrogatory 
example is then generated. The procedure is applied 
iteratively until either the quantity l/(l-»-R) exceeds the 
beta value, or all interrogatory examples in the pool for the 
concept have been presented. 

In the MAIS decision procedure, it is assumed that the 
form of the loss structure involved is a threshold function. 
Therefore, only the loss ratio R has to be assessed 
empirically. However, a decision-maker's loss structu^ can 



(6) 



TQ = arcsin/fTQ 

gjf = arcsin((a+x+3/8)/(a4-p+n+3/4)) 
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be completely assessed without making any assumptions about 
the form of the loss function. Only minimal axioms from 
utility theory have to be assumed. However, as van der Linden 
(1981a) has pointed out, these techniques do not 
automatically lead to elegant loss functions and optimal 
cutting scores. It may be wise, therefore, to use these 
techniques only for a priorly chosen mat-hersatical form of the 
loss function. In addition to the threshold loss function, 
however, mcr<^ useful functions have been adopted in decision 
theory. One such function will be considered below. 



The Linear Loss Model 

An obvious disadvantage of the threshold loss function is 
that it assumes constant loss for students to the left or to 
the right of wq, no matter how large their distance from vq. 
For instance, a misclassified true master (see Table 1) with 
a true level of functioning just above itq gives the same loss 
as a misclassified true master with a true level far above 
TQ. It seems more realistic to suppose that for misclassified 
true masters the loss is a monotonically decreasing function 
of the variable ir. 

Moreover, as can be seen in Table 1. the threshold loss 
fxinction shows a "threshold" at the point ir = ttq, and this 
also seems unrealistic in many cases. In the neighbourhood of 
this point, the losses for correct and incorrect decisions 
frequently change smoothly rather than abruptly. 
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In view of this, van der Linden and Mellenbergh (1977) 
propose a linear loss function: 



(8) Kai.TT) = 



bo(Tr-TrO)+dO for i = 0 (retain) 

bo. bl > 0. 

bi(TrO-ir)+dl for i = 1 (advance) 



The above defined function consists of a constant term 
and a term proportional to the difference between the true 
level of functioning ir and the specified criterion level ttq. 
The constant amount of loss, dj (J =0,1), can, for example, 
represent the costs of testing. The condition bg, b^ > 0 is 
equivalent to the statement that for actions ao and aj^, 
..tility is a strictly increasing and decreasing function of 
the variable ir, respectively. The parameters bg. bj^, dg. and 
d^ have to be assessed er^pirically . Figure 1 displays an 
example of this function. 



Insert Figure 1 about here 



The linear loss function seems to be a realistic 
representation of the losses actually incurred in many 
decision making situations. In a recent study, for example, 
it was shown by van der Gaag (1987) that many empirical loss 
structures could be approximated by linear functions. 
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As the general linear loss fimction now stands, we need 
to determine the four constants bg, bi, dQ, and d^ before it 
can be applied. However, if we use the fact that a loss 
function needs to be determined only up to a positive 
multiplicative and additive constant (e.g.. Luce & Raiffa, 
1957), we can reduce the munber of unloxown constants to two. 
Thus, since b^ > 0, we may redefine l(aj^,ir) by making the 
positive linear transformation l*(ai,Tr) = [l(ai,Tr)-di] /b^ . 
And so 



(9) l*(ai,ir) = ' 



b*(Tr-TrO)+d* for i 



¥0 - IT for i = 1. 



where b* = bQ/bj and d* = (do-di)/bi. 

We turn now to an illustration of one of the most direct 
methods available for determining the constants b* and d* . In 
order to make the method work, the decision-maker must be 
able to specify two ordered pairs (Tri,Trj) and (Tri.TTj) s*ich 
that 

l*(ao.Tri) = l*(ai,¥j) 
(10) and 

l*(ao.Tri) a l*(ai,¥j) . 

Solving this system of equations, we find that 



(11) b"^ = (TTj - Tri)/(Tri - ^i) and d* = ttq - ttj - b*(Tri - ttq) 
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Choosing the ir-coordinate of the intersection of both loss 
lines l*(ao,ir) and l*(aj^.ir) as one of the ordered pairs, it 
follows that 

(12) ^ ^ ^ 
d* = (Tro-Trp)(l+b*). 

where irp denotes the point of intersection. 

The decision rule that minimizes the posterior expected 
loss in the case of a linear loss function is to advance a 
student with test score x for which 

(13) E[(¥o-Tr) |x,n] ^ E[b*(Tr-Tro)+d* |x,n] 

and to retain him/her otherwise. Since (l-fb*) > 0. this is 
equivalent to advancing a student if 

(14) E[Tr|x,n] ^ ttq - dV(l+b*) . 

and retaining him/her otherwise. In other words, with linear 
loss, the action taken depends only upon the expectation of 
the posterior distribution of ir. other attributes of the 
distribution are irrelevant for decision purposes. 

Using the fact that the expectation of a beta 
distribution B(a.p) is equal to a/(a-^p). and. thus, the 
posterior expectation equals (a+x)/(a+x+p+D-x) , it follows 
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that a student is advanced if his/her test score x is such 
that 

(15) [o+P+n] [Tro-d*/(l+b*)] - a. 

and retained otherwise. 

Putting l*(ao.Tr) and l^Ca^.Tr) equal to each other, it 
appears that the ir-coordinate of the intersection of both 
loss lines from Formula (9). iTp. is equal to ttq - d*/(l+b*). 
Therefore, the decision x'ule can be viewed as advancing a 
student if his/her expectation of the posterior distribution 
of IT is to the right of the intersection point, and retaining 
him/her otherwise. Note that with linear loss, only the point 
of intersection, irp, of the two loss lines for retain and 
advance are needed, and, thus, the intercept and slope of 
Formula (9) does not have to be estimated. Hence, expression 
15 is equivalent to 

(16) X ^ [o+p+nJiTp - a. 

When d* = 0, that is, dg = d^, both loss lines intersect 
at ir = iTQ and an interesting case arises. Then, all loss 
function parameters vanish from the decision rule and, thus, 
it takes the form of advancing a student if 

(17) E[Tr|x,n] z ttq, 
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and retaining him/her otherwise. In other words, if the 
amounts of constant loss* dj (j = 0,1), for both decisions 
are equal, or if there are no constant losses at all (i.e., 
no costs of testing are involved) , then there is no need to 
assess the parameters d* and b* in adapting the number of 
interrogatory examples. In that case, the decision rule can 
even be simplified to advance a student if his/her 
expectation of the posterior distribution of ir is greater 
than or equal to the specified criterion level itq, and to 
retain him/her otherwise. 

In HAIS, it is assumed that the form of the psychometric 
model relating observed test scores to student's true level 
of functioning can be represented by the binomial model 
(Equation 1). In the next section, another psychometric model 
used in criterion— referenced testing will be considered. 



Classical Test Model 

The expectation of the posterior distribution, E[ir|x,n], 
represents the regression of ir on x. A possible regression 
function is the linear regression function of classical test 
theory (Lord & Novick. 1968): 

(18) E[Tr|x,n] = pxx' + d-PxX') — ' 

n n 

M'X and Pjx' being the mean and reliability coefficient of X. 

Er|c 2i 
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Equation 18 is known as Kelley's regression line. This 
IS an interesting equation in that it expresses the estimate 
of the true level of functioning as a weighted sum of two 
separate estimates - one based upon the student's observed 
score, X, and, the other based upon the mean, of the 

group to which s(he) belongs. If the test is highly reliable, 
much weight is given to the test score and little to the 
group mean, and vice versa. 

Substituting Equation 18 into expression 14, and solving 
for X gives the following optimal seG[uential decision rule 

n{iro - d*/(l+b*)) -^x 

(19) X i jix + . 

PXX' 

If the amounts of constant loss for both decisions are equal, 
i.e. d* = 0, or if there are no constant losses at all, 
expression 19 will take the rather simple form 



(20) X 2t Jix + . 

Ph- 



ot 



(21) X z 



PXX' 



Since 0 ^ pjx' ^ 1 . and, thus -1 ^ PXX'^^ ^0, it 
follows from expression (21) that ^x optimal 
sequential cutting score, x, are related negatively. The 
higher the average performance, the lower the optimal 
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sequential cutting score. Hard-working students are rewarded 
by low cutting scores, while less hard-working students will 
Just be penalized and confronted with high cutting scores. 
This is the opposite of what happens when norm-referenced 
standards are used (van der Linden & Mellenbergh. 1987). 

It should be stressed that, as can be seen from (20), 
the optimal sequential cutting score. I.e. the number of 
Interrogatory examples to be administered ^o the student, 
depends upon and Pxx'* decision cotnponent In HAIS 

allows for an updating after each response to an 
Interrogatory example. This explains why. though the 
decisions for determining the optimal number of 
Interrrogatory examples are made with respect to an 
Individual student, the rules for the declsons are based on 
data from all students taught by the system In the past and. 
in doing so. are Improved continuously. In other words, a 
computer-based adaptive Instructional system can be designed 
in this way. I.e., a system of rules Improving Itself over 
the history of the system as a result of systematically using 
accumulated data from previous students. The parameters of 
the model, [ij and PxX'* updated each time a next student 
has finished his/her dialogue with the system. Eioball (1971) 
entitles such systems as "self-optlmlzlng" . 
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Characteristics of Adaptive Instruction 
in the Extended MAIS 

In this section, the six defined characteristics of 
computerized adaptive instruction according to Tennyson and 
Park will be discussed for HAIS with the extensions proposed 
in the present paper. 

The first characteristic is initii.1 diagnosis and 
prescription. MAIS uses this characteristic by administering 
a pretest (pretask data) to all students from which an 
initial prior estimate of a student's ability results. The 
prior distribution is supposed to be a beta distribution. 
B(a.p}. Estimates of the parameters a and P are given by 
Equation 5. As an aside, it may be noted that if 
administering a pretest is not possible for any reason, the 
prior distribution of a student can be characterized by a 
un.lform distribution on the interval from zero to one. In 
that case, the parameters of the beta prior should be 
specified as a = p = 1. Also, the use of Kelley's regression 
line (Equation 18) as an estimate of the true score can be 
considered as an informal way to take account of prior 
knowledge . 

The second characteristic concerns the sequential 
character of the instructional decision-making process. MAIS 
uses this characteristic when updating the beta value after 
each interrogatory example (on-task data) and comparing this 
value to the quantity 1/(1+R) (Expression 3). Using a linear 
loss function, the posterior expectation of the true level of 
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functioning should be compared with the point of intersection 
of the two loss lines for the retain and advance decisions 
(Expression 12). Moreover, when the amounts of constant loss 
are equal or, if there are no constant losses at all. the 
sequential decision rule can even be simplified to comparing 
the posterior expectation with the criterion level, itq. set 
in advance (Expression 17) . Finally, it has been indicated in 
the previous section that the sequential rules for the 
instructional decision making process can even be based on 
data from all students taught by th*> system in the past, if 
we adopt the classical test model as the psychometric model 
actually involved (Expression 19). 

The third characteristic refers to the appropriate 
amount of instruction each student receives to achieve the 
defined instructional objectives. As mentioned before, in 
HAIS this is done by determining the optimal number of 
interrogatory examples. Besides the threshold loss function 
assumed in H&IS. the optimal number of interrogatory examples 
has been determined in ^ase a linear loss function as well as 
lelley's regression line of classical test theory is adopted. 

The fourth characteristic pertains to the sequence of 
instruction. In MAIS this characteristic follows closely a 
cognitive theory of concept and rule learning. Since this 
characteristic is not controlled by the decision component, 
we refer for further discussion to Tennyson. Youngsters and 
Suebsonthi (1983). 

The fifth characteristic is instructional time control. 
Control of the instructional time is associated directly with 
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the Bayesian beta values established for the necessary amount 
of instruction. The proposed extensions of MAIS described in 
this paper, namely the linear loss model and the classical 
test model, can ba applied to control the instructional time 
in a possibly even better way by using the improved optimal 
sequential decision rules. 

Finally, the last characteristic concerns the advisement 
of learning need. This advisement provides the necessary 
information by which students can make adequate self- 
assessments and learning needs Judgment. As noted before, 
this characteristic is one of the eight instructional design 
variables within MAIS and is also directly managed by the 
decision component. Analogous to the previous characteristic, 
it night be expected that by using the proposed linear loss 
model and the classical test model, this characteristic can 
be used in such a way that MAIS can even be made inore 
effective. 

In summary, five out of th« six characteristics 
identified as essential in an effective computerized adaptive 
instructional system can be improved by applying the 
extensions discussed in this paper. 

New Lines of Research 

There are a few new lines of research arising from the 
application of psychometric theory to the decision component 
in MAIS. The first is the extension of the work of Tennyson 
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and his associates to situations where guessing and 
carelessness are incorporated. Morgan (1979) has developed a 
model with corrections for guessing an.d carelessness within a 
Bayesian decision-theoretic framework. The results of a 
computer simulation of the model indicate that guessing and 
carelessness may markedly affect nhe determination of cutting 
scores, and hence the accuracy of decisions about mastery. 

The second line is research into other prior 
distributions for ir (for example, the standard normal 
distribution) than the beta distribution assumed in H&IS. It 
might also be assumed that no prior distribution about t is 
available, because specifying such a distribution i? too 
difficult a Job to accomplish. In these circumstances, the 
minimax procedure may be an appropriate framework (e.g.. 
Huynh. 1980; van der Linden. 1981b) which requires no prior 
distribution regarding the true level of functioning. In this 
case, the optimum cutting score is obtained by minimizing the 
maximum risk which would incurred by misclassifications. As 
an aside, it might be noted that a minimax rule can be 
conceived as a rule that minimizes the "Bayes risk" as well, 
but uuder the restriction that the prior is the least 
favorable of the c3ass of priors (e.g.. Ferguson. 1967. 
sect .1.6. ) . 

The third line of research concerns still other loss 
structures than the threshold and linear loss function, which 
represent possibly even more realistic forms that loss 
structures might take in applications Co instructional 
deci.iion making in computer-based adaptive instructional 
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systems. For example, the normal ogive function (Novick 4 
Lindley. 1978) which takes loss to be a nonlinear ly fxmction 
of the true level of fxmctioning This loss fxmction does 
not only have realistic properties but also can be combined 
nicely with a normal model for the test data. 

Finally, an interesting new line of research seems to be 
an extension of the actions available to the decision-maker. 
In MIIS. two actions were available to the decision-maker, 
namely advancing (a^) or retaining (ag) a student. However, 
it might also be assumed that there are three (or any finite 
number) of actions open to the decision-maker. For example, 
in the three-action problem the student may provided with 
additional instructional materials both of the present and 
the previous concept (a2); (s)he may provided only with 
additional instructional materials of the present concept 
(ag); or. (s)he may advance to the next concept (aj^). 

He might think of this problem in terms of specifying 
two cutting scores eg and c^ on the observed test score scaJ^j 
X. where cg < cj. Then for observed test score x < cq. action 
a2 will be taken; for cq < x < c^. action ao will be taken: 
and. for x > c^. action a^ will be taken. 

Davis. Hiclcman and Novick (1973) have given a solution 
to the three-action problem using a natural extension of the 
threshold loss function. Although the notation becomes more 
complex and the computation a bit more tedious, there are no 
fundamentally new ideas in the multiple-action problem. 
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Some Suomary Remarks 

In this paper it was indicated how the M&IS decision 
procedure could be formalized within a Bayesian decision- 
theoretic framework. In fact, it turned out that this 
decision can be considered as a sequential mastery decision. 

Moreover, it was argued that in many situations the 
assumed threshold loss function in MAIS is an unrealistic 
representation of the loss actually incurred. Instead, a 
linear loss function was proposed to meet the objections to 
threshold loss. 

Further, Eelley's regression line of classical test 
theory was proposed as the psychometric model relating 
observed test scores to the true level of functioning. Using 
this psychometric model instead of the binomial model assumed 
in MAIS, computer-based adaptive instructional systems can be 
designed in which the determination of the optimal number of 
interrogatory examples for an individual student is based on 
data from all students taught by the system in the past. 

With these two extensions of the MAIS model, it might be 
expected that five out of the six characteristics of 
computerized adaptive instruction can even be improved in 
this model. 

Whether or not the proposed linear loss model and the 
classical test model are, however, real improvements of the 
present decision component in MAIS (in terms of student 
performance on posttests, learning time, and amount of 
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Instruction) oust be decided on the basis o£ empirical 
experiments . 
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Author's Note 

The author is indebted to Wim J. van der Linden for his 
valuable cooments on earlier drafts of the paper. 
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Table 1 

Twofold Table for Threshold Loss Function 
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Figure Caption 

Figure 1. An Example of a Linear Loss Function, 
(bo ^ bl , do di) 
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